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The other papers in the symposium have concentrated on t He effects of 
errors of measurement on various types of correlation measures. In contrast, 
the purpose of the present paper is to consider the properties of ANOVA, 
regression anilyses, ANCOVA, and factor analyses when employed on fallible 
variables, i,e t| variables which contain errors of measurement . Because 
of the broad range of topics to be considered, the discussion of each will 
be somewhat brief. For each topic the major problems of concern to the 
user of statistics will be identified; when available, solutions to the 
problems will be indicated; and several of the more important references 
will be cited. 

Factur analysis is considered first because the primary concern is 
with the properties of the elements in the correlation matrix to be factored, 
and so perhaps will facilitate a transfer from the preceeding papers. The 
topic of regression analysis is considered second because it logically leads 
the way to subsequent discussions of ANOVA and ANCOVA. As Fisher (1932) has 
said, ANCOVA “combines the advantages and reconciles the requirements of tne 
two very widely applicable procedures known as regression and analysis of 
variance . " 

Factor Analysis 

Although the literature abounds with articles on the theory and use of 
factor analysis, few have been concerned with the nature of the correlations 
in the matrix to be factored. When discussing the nature of correlation co- 
efficients in a matrix to be factored, it is useful to make a distinction 
between manifest and latent relationships. Manifest relationships are those 
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obtained from variables as they are observed, while latent relationships are 
those which "may be- inferred to exist between variables and which are masked 
or distorted by various kinds of errors and constraints" {Carroll, 1961, 
p, 151), When used as an instrument to facilitate theory building, factor 
analysis should operate on a matrix of correlation coefficients which reflect 
the latent relationships among the variables, 

Cavroll (1961) has identified the following four categories of errors 
and constraints which can affect the value }f a Fearson Prod uc t -Moment cor- 
relation coefficient: 

1. errors of scaling, 

2. errors of scale-dependent selection, 

3. scedastic errors of measurement, 

4. topastic error.*? of measurement , 

Errors of scaling result when a dichotomy is forced on a continuous variable. 

An error of scale-dependent selection occurs when the sample is taken such 
that subjects with extreme scores on either or both variables are not selected. 
Scedastic errors of measurement are those dealt with in classical measurement 
theory, i.e. independent of each other and the latent variable with expected 
value zero; and topastic errors of measurement are created when a subject 
guesses correctly on a multiple choice item. 

Carroll states that except for scedastic errors of measurement the 
errors and constraints can alter the rank of a correlation matrix and cause 
subsequent factor analysis to yield spurious results. The exception made for 
scedastic errors probably stems from results given by Roff (1937) and later 
supported by Saunders (1948) which indicate that 

1. the rank of the correlation matrix, R, is unaffected by 
scedastic errors; 
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2. the communall’iy of an infallible variable is equal to the 
communality of the fallible variable divided by the relia- 
bility of the fallible variable; 

A 

and 3. the factor pattern matrix of the infallible variables, F , 
is related to t lie factor pattern matrix of the fallible 
variables, F, by the equation 

* 

F = AF, 

where A is a diagonal matrix of the inverses of the 
square roots of the reliabilities of the fallible 
variables . 

As will be pointed out later, the exception made for scedastic errors was 
a mistake since the relationships given by Roff are based on the seldom met 
assumption that communal i ties are known rather than estimated. 

A matrix of Pearson Product-Moment coefficients for variables containing 
errors of scaling generally results in a factor analysis solution which con- 
tains a difficulty factor, although Kaiser (1970) has recently indicated that 
Guttman's image analysis may not. Carroll (1961) recommends the use of tetra- 
choric coefficients to side step the problem of errors of scaling. The problems 
of errors caused by scale-dependent selection have not been solved except by 
the obvious method of avoiding them via careful sampling procedures. The 
procedures of correcting correlations for restriction of range (Guilford, 1959; 
Bryant, 1970) Might be useful, but to my knowledge the consequence of using 
such corrected coefficients in factor analysis have not been investigated. 
Carroll (1961) has given a method of correcting joint and marginal distri- 
butions of the observed variables for topastlc error, and ruggei ts that cor- 
relations be calculated on the corrected distributions. 

Glass (1966) has indicated the problems with several methods of factor 
analysis which result from scedastic errors. In particular he has shown that 
a components analysis of the correlation matrix with ones in the main diagonal 
and off diagonal elements corrected for attenuation is not necessarily of the 
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same order as components analysis of the uncorrected matrix, factoring 
the correlation matrix with GuLtman's lower bound to communal [ t tos Ln the 
main diagonal does not result in the relation F* - AF, nor does Rao's 
canonical factor analysis. On the positive side, Glass demonstrated that 
F* = AF for Kaiset^s alpha factor analysis. 

3rtefly, alpha factor analysis is an iterative procedure which starts 
with a principal axis factorization of the matrix 
Hj-l <R • I)H _1 + I, 

where is a diagonal matrix of the multiple correlation of each variable with 
the remaining variables and I is the identity matrix. As many factors are re- 
tained as there are latent roots of the above matrix which exceed one. The 
solution is used to calculate new estimates of the communalit ies which are 
used to replace those in to yield The procedure is repeated until 

the estimates of the communalit ies have converged according to an arbitrary 
apriori ciiterion. Glass has shown that by replacing with H^l “ ^ * 

alpha factor analysis results in the relationship F* = AF provided that and 

will iterate to the same matrix of communalit ies . A necessary assumption 

2 2 2 

is that the elements of A satisfy hj <_ 1/a^ £ 1 for all j, where hj is the 
communal ity of the j bh variable. Glass has demonstrated on several well 
known examples that the two matrices do converge on the same parameters. Glass 
(1966, p, 559) further derived that "Alpha factor analysis applied to fallible 
and infallible variables separately will be equivalent in terms of number of 
factors, complexities of corresponding variables, and patterns of simple struc- 
ture"; "Normal varimax rotations of F and F* will yield derived solutions F^ 
and F^* such that AF^ = F^*; and "The alpha factor scores for the fallible and 
corresponding infallible variables may be considered identical*" 
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The above suggest that a researcher interested in using factor analysis 
to investiage the latent structure of variables should first be careful in 
his sampling of subjects to avoid possible problems brought on by errors of 
scale-depend^ut selection. Second, he should employ Carroll's correction 
to the marginal and joint distributions of each pair of variables >_o control 
for topastic errors. Third, he should calculate tetrachoric coefficients on 
the corrected distributions to avoid the problem of difficulty factors arising 
from errors of scaling. Finally, given the resulting correlation matrix he 
should use Kaiser's alpha factor analysis to side step the problem of scedastic 
errors of measurement. The use of alpha factor analysis reems to follow 
Kaiser's recently stated first principle in dealing with problems of factor 
analysis, i.e. "It don't make no never-mind." What is meant by the principle 
he says is "that when faced with a crucial decision, don't try to settle it; 
rather, avoid it!" (Kaiser, 1970, p. 403). 

Regression Analy ses 

Madansky (1959) identifies three basic types of regression relation- 
ships which arc generally referred to, although not always (Lindley, T947), 
as regression, structural, and functional. Regression is defined as the 
appropriate relationship for predicting one set of scores from another. 

Because the purpose is to predict one set of scores on the basis of another 
set of scores, the relation should be defired by the observations, making 
the least-squares estimate appropriate, i.e*, the manifest relationship is 
of interest. A structural relationship is defined by the true, parts of the 
variables, when thu independent variable is random. A functional relation- 
ship is also defined by the true parts of the variables, but the independent 
variable is fixed, and the true variables are perfectly correlated. These 
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last two types of relationships fall under the general category of latent 
relationships and are of interest when theory building or testing is the 
objective. Because problems of prediction and the regression relationship 
are familiar, and because functional relationships are probably quite rare 
in educational research, the following discussion emphasizes structural 
relationships. Both Madansky (1959) and Cochran (1968) offer excellent 
reviews of the work done on estimating structural relationships. 

First mention of the inappropriateness of a least-squares estimate of 
the structural relation, when the variables are fallible, was made by 
R. J. Adcock (1878), However, in an early review of the problem of esti- 
mating structural relations, Roos (1937, P, 7) credits Corrndo Gini in 7921 
as the first to recognize that "if the errors of X and Y are independent, 
then the least-squares 0 is larger than the 0 of the actual line of best 
fit," Since then, a considerable body of literature has dealt with the 
problem of estimating the structural relation when both variables are 
fallible. 

In the first half of a paper by Berkson (1930), the problem of esti- 
mating a structural relation is stated, and an analytic demonstration of 
the bias of the least-squares solution is given for the case of scedastic 
errors of measurement. Berkscn's demonstration shows that the 0 defined 
by a fallible dependent variable, Y, and a fallible independent variable, 

X, is equal to the 3 defined by the true parts, multiplied by the ratio 
of the variance of the true parts 0 f X over the variance of the true parts 
of X plus the variance of the error parts of X, In the notation adopted 
here 
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where X denotes the true parts of X. The ratio which defines the bias of 
the least-squares (3 is equivalent to the measurement theory definition of 
the reliability of X (Gulliksen, 1950, p. 25). Thus 

^Y.X = |0 XX B Y.T ’ 

where denotes the reliability of X. It should be noted at this point 
that the bias does not depend on the dependent variable, Y, and therefore, 
the fallibility of Y does not affect the least-squares estimate of tie 
structural relation. Rather than offering Berkson’s derivation, a deri- 
vation which is consistent with measurement theory seemed more appropriate. 

Let p YT be the correlation of Y and T, and p^y be the correlation of 
Y and X. 

6 Y.X = P YX — - — » 

a x 

but p YX = Py T f (Gulliksen, 1950, p. 105) 

and - J T r 



Therefore, 6 y „ = P yT P xx °Y 



and 6 Y.X = P XX B Y.T' 

Because the expected values of the fallible variables are equal to the expected 
values of the infallible variable 

°Y*X * ^.T + ^Y«T ~ 

where a defines the Y intercept of the regression line. 

Under the same classical model for errors of measurement but in dif- 
ferent notation, Cochran (1970) gives the relationship between the manifest 
and structural multiple regression equations as 

- ^ (l-py y ) 3\ 
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where i=l, k denotes which independent variable. The relationship 

between the manifest and structural. Intercepts is 



- n + >: 
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1*1 



Y*T, 
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where a* denotes the manifest intercept and a denotes the structural intercept. 
For V~1 the above relationships reduce to those given earlier for a single »' 

i 

independent variable. Cochran concludes that the direct effect of errors in 

t 

t 

an independent variable is to decrease the absolute value of its associate i 
8 weight by at least a factor of its reliability coefficient, Cochran ( 1 9 / 5 8 ) 
has also given a statement of the above relationships under a slightly lei s 
restricted model for errors of measurement. Lindley (1947) has demonstrated 
that even when the structural relationship is linear, the manifest relation- 
ship need not be linear. If the infallible variables are multivariate nor- 
mal then the errors of measurement must also be multivariate normal to assure 
a linear manifest relationship. 

Karl Pearson (1901) offered the first approach to gain any prominence 
for estimating structural relations, using fallible variables. Pearsor 
proposed minimizing the sum of the squared normal deviates of the observed 
points from a line which has come to be called the orthogonal regression 
line. Allen (1939) has shou'rt that an orthogonal regression line is d'lpen- 

t 

dent upon the choice of units used in measuring the variables. The ortho- 
gonal regression line can always Le the structural relation, If the rciits 
chosen to measure the variables happen to be the right ones. There ;.s no 
practical way of determining what the units should be without additional 
information (Allen, 1939, p. 198). C. F. Roos (1937, p. 18) offers -an 
orthogonal regression line solution, which is invariant to the metr’.c of 
the variables but requires apriorl information about the magnitude of the 
O rrors. \ 

me 
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Most of the developments since Pearson can be placed in one of the 
four categories: the method of grouping, the use of instrumental vari- 

ables, the use of variance components, and the Berkson case. To supply 
detailed information about tha various types of estimates would not be 
appropriate here, but a brief acquaintance with one of the more popular 
techniques in each category should provide helpful background. 

Wald (1940) provides a method for estimating structural relations, 
based on the method of grouping. Letting X denote the independent variable 
and Y the dependent variable, Wald makes the following assumptions: 

1. Errors on X are unconclated and have a common distribution, 

2. Errors on Y are uncorrelaled and have a common distribution, 

3. Errors on X are uncorrelated with errors on Y, 

4. There is a single linear relation between the true variables, 

5. Observations on X, can be divided into subgroups in such a 
way that the true part of any observation will be in the 
same subgroup as its associated observed score. 

The location of the line, identifying the structural relationship, is 

estimated by the mean coordinates. To estimate slope, Wald first orders 

the observations on X and divides them into two g' oups of equal size. The 

joiu of the mean coordinates of the tvo groups provides the estimate of 

slope. Wald derived the variance of the above estimate of a structural 

relation and provided a test of hypotheses about the size of the slope. 

Wald demonstrated that his estimate of structural relation is consistent 

estimate and that the hypothesis test is exactly correct if the assumptions 

are satirfied. 

Barllett (1949) extended Wald’s estimate by proposing that the sub- 
groups for estimating slope be the upper and lover one-third of the ordered 
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X values. Bartlekt provides confidence interval estimates for his estimate 
of slope and shows that, when X has h rectangular distribution, the esti- 
mate is more efficient than Wald's. More generally, the efficiency of a 
criterion for grouping is dependent upon the distribution of the indepen- 
dent variable. far a normally distributed independent variable, Madansky 
(1959) has shown that the most efficient criterion for grouping is to use 
the upper-lower twenty-seven percent. Madansky (1959, p . 184) offers a 
table, which indicates the most efficient criterion for grouping several 
different types of distributions. Besides being dependant upon the dis- 
tribution of the independent variable, the method of grouping further 
requires knowledge that the errors of the independent variable are in- 
dependent of tie grouping. Without this additional information, the 
estimates are not necessarily consistent (Neyman, 1951) . 

Reiersol (1945) originated the strategy of using instrumental 
variables in tie estimation of structural relations ani the approach has 
since been developed by others. An approach, taken by Durbin (1954), 
represents the simplest use of an instrumental variable, although probably 
the least often applicable. The assumptions are the tame as the first 
four giv n, iu regard to the method of grouping. Let z be an infallible 
instrumental variable which is correlated with the true parts of the 
independent variable but not with the error parts. Then 



; di Y i 

i»l 



£ zX 
i=l 



is a consistent estimate of the structural relation of Y on X. Durbin offers 
a proof of b's consistency and derives a confidence region for the parameter 
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of the structural relation by making use of the fact that true regression 
Y - 8 on z is zero. As would be expected, Durbin's ej titnate is less 

A 

efficient than a least-squares estimate, based on the unobservable true parts 
of the fallible variables. 

Reiersol (1945) has considered the estimation of structural relations, 
using fallible inscrumental variables. Reiersol 1 s estimate requires obser- 
vations on two instrumental variables, both of which are correlated with 
the true parts of the independent variable, but not with the 6 :^*or parts, 
and further states that these instrumental variables have some known 
linear relation. 

Geary (1949, p. 30) states that the accuracy of using instrumental 
variables to estimate structural relations is dependent upon the correlation 
of the instrumental variables with the dependent and independent variables. 
Even if instrumental variables can be identified as having the required 
properties, which in itself seems unlikely, their use represents* additional 
cost. Madansky (1959, p. 188) identifies Durbin’s use of an infallible 
instrumental variable as equivalent to the grouping method, when the instru- 
mental variable is restricted to the values -1, 0, and +1. 

The variance components procedure for estimating structural relations, 
using fallible variables, began with Tukey (1951), and requires additional 
information, similar to that necessary for grouping. The data must be in 
the form of observations, X^ , on each of N X^'s, i.e», two or more 

groups must be identified into which the data iray be divided. The pro- 
cedure is to do one-way analyses of variance on the variables X, Y, and 
XY, as in an analysis of covariance* The mean squares and expected mean 
squares of the computations are given in Table 1. (Madansky, )939, p» 189). 
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An inspection of the expected mean squares in Table I indicates 
b" r Ilf - VI 

N I - iv 

is an estimate of the structural relation of Y on X, Tukey show? the estimate 
to be consistent as N ■+ » and seme n i -> and provides confidence intervals 
for 3, both in the case cf a functional reJ . *n and in the case of a struc- 
tural relation. 

Berkson T s method (1950) for estimating structural relations is dependent 
upon his sampling model which distinguishes between a ’'controlled observation” 
and an "uncontrolled observation." Let x = X + u be an observation on the 
fallible independent variable, broken down into a true part X and an error 
part u; and similarly, let y = Y ^ v be a fallible observation on the depen- 
dent variable. An "uncontrolled observation" on the fallible independent 
variable is defined as X being fixed, and u being a random variable, inde- 
pendent of X. The structural relation is Y - a + 3X and by substitution 
y = a + B x + (v - Bu)» The above equation is not a standard regression 
model because the random error (v - 3u^ is not independent of x. The 
purpose of taking a "controlled observation" is not to estimate X, but 
rather, to bring the observed quantity to a set value. The. attempt is 
to get x each time an observation is made, but because of errors of 
measurement, the observation represents x - u. The sampling model causes 
X to vary whi)e x remains fixed; therefore, u is independent of x. Since 
for a "controlled observation" the error of measurement is independent of 
the obse^vat ..vii, the least-squares estimate of slope is also an unbiased 
estimate of the structural relation. A fixed x also eliminates the popu- 
lation correlation of x and y, which means the regression line y on x is 
the sama as the regression line x on y. Scheffe (195&) offers a slight 
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TABLE 1 

Tukey's Variance Components* 



Source 



Mean Square 



Expected Mean Square 



II 



III 



IV 



VI 



N - . 2 

£ n i (X i --X B- ) / (N-l) 



i= 1 
N 
i=] 



1 n i(X it -X t .X^i.-Y^) 



/(N-l) 



" n i (Y i “ Y ..> t (N-l) 
i-l 



N ni _ , 

I E (X^-Xi )* / (N-l) 

i-.l J = I 

N 11 i _ _ 

l l (X^-X, ) (Yj ,-Yj ) 

i-l j=l J J 

/(N-I) 

N n i 

E E (Y^-Y,.) 2 / (N-l) 

1=1 j=l J 



2 2^2 2 
a + IN - l nj / (Nn-a)J o 

u 1-1 1 



2 N 2 

cov(u.v) + t(N - E a.) / 
1=1 



(Nn-ti) ] 00 



o 2 + [(N 2 - E n 2 )/(Nn-n)]8 2 o 2 
v i=1 i * 



cov(u,v) 



u - error part: of X 
v ■ error part of Y 
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modification of Berkson's model by allowing u and v to be independent 
variables, with variances and The modification is dependent 

upon the manner in which replication is achieved. Scheffa supposes that 
replicates are taken by changing the controlled bariable from its pre- 
vious value and bringing it back for each replicate; whereas Berkson 
assumes the controlled variable remains unchanged. Scheffe develops 
confidence intervals for the slope and intercept, under the assumptions 
of his restatement of the Berkson case. 

ANOVA and ANCOVA 

The effects of scedastic errors of measurement in ANOVA are the same 
as they are in ANCOVA with a fallible dependent variable and an infallible 
covariable. Because scedastic errors have an expected value of zero and 
are independent of the true parts cf the variable, the least squares pro- 
cedures of ANOVA provide unbiased estimates of the parameters in its linear 
model. In ANCOVA the least squares estimate of the slope of the structural 
relation of the dependent variable on the covariable is unbiased if the co- 
variable is infallible, as seen earlier, and so the least squares procedures 
of ANCOVA also provide; unbiased estimates of the parameters in its linear 
model. Cochran (1968) has considered the problem of estimating the parameters 
in the linear model o>: ANOVA for a less restrictive model cl errors of measure- 
ment and comes to the same favorable conclusion. 

Given the usual assumptions, the F test statistics for ANOVA on a fal- 
lible dependent variable and the F t?st statistics of ANCOVA on a fallible 
dependent variable and an infallible covariable will follov. theoretical F 
distributions. The only detrimental effect that scedastic errors of measure- 
ment in the dependent variable have is to decrease precision and thus increase 
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the probability of a Type II error. For ANOVA the decrease in precision is 
seen by the equation 



where a ^ denotes the variance of the fallible dependent variable, 0 ^ is the 

variance of the infallible dependent variable which is inflated by a 2 the 

e 

variance of the errors of measurement. The error variance of a one-way 



AN CO V A is 

% <i - 4> i 1 + -iV- 1 ’ 

e 

2 

where 0 ^ is the variance of the fallible dependent variable, is the cor- 

relation of the fallible dependent variable with the infallible covartsble 

2 

and f denotes the degrees of freedom for estimating error variance. The o v 
e i 

term is inflated by errors of measurement in the same way as shown above for 
ANOVA. Further, p ^ is attenuated by errors of measurement in Y. Both 
effects cause a loss in precision due to a fallible dependent variable. 
Sutcliffe (1958) has derived the expected mean squares for a one-w»ay, fixed 
effects ANOVA for a fallible dependent variable. By using his table of ex- 
pected mean squares, Sutcliffe points out the increase in the probability 
of a Type II error caused by the errors of measurement, iox (1961) has 
considered whether the ANOVA least squares methods employed on fallible 
variables provide unbiased estimates of error variances in factorial and 
fractional factorial designs. He concludes that the estimates are unbiased 
for both high order interactions and the pooled variance of replications 
within ^clls, 

The use of a fallible covar.iable in ANCOVA can causa a far more dis- 
tressing problem than decreased precision. First, consider the linear model 
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of a one-way fixed-effects ANCOVA with a single random covariable. Note that 
a random covariable represents a relaxing of the restriction in the classical 
model that covariables be fixed. DeGracie (1968) has shown that the usual 
ANCOVA procedures applied to data with a random covariable still provide un- 



biased estimates of the parameters in the linear model as well as valid test 
statistics. The only difference from classical results is that the variances 
of the various estimates are averaged across all values of the covariable. A 
random covariable is certainly more representative of practice in educational 
research than is a fixed covariable. The linear model is 

Y ij = V Y.. + + Vx (X ij " V X.. J + C ij ’ 



where Y^j and are the i th_ observations in the j th treatment for the 
dependent and covariable respectively, 

y Y is the constant for true mean response, 

n j = Py . - h y - 3y tX (p x j “ ) is the j th treatment effect. 



6 



Y • X 



is the common within treatment slope of 1:he regression of Y on X, 



and are random variables assumed to be normally distributed independent 

of each other with zero mean and comm', variance. 

When ANCOVa is employed as a method for gaining precision In designs 
where experimental units are randomly assigned to treatments and where the 
covariable is observed antecedent to the experiment, y = y for all 

x .j x o' 



j i j 1 , and the treatment effects, a y reduce to those for ANOVA, i.e. 

My - V v . The only negative effect of a fallible covariable beyond that 
“■ » Y • • 



of a fallible dependent variable is a further attenuation of causing a 

AY 

lose, in precision as seen earlier. However, when ANCOVA is used in attempt 
to control for systematic initial between group differences on the covariable, 
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in general u ¥■ p v and the treatment effects will reflect the nature 

*.j o’ 

of the correction term 



8 Y-X (y x J ” M x.h* 

This second use of ANCOVA has become common practice in educational research 
since suggested by Campbell and Stanley (1963) for use on their quasi- 
experiments. Lord (I960), Smith (1957) and Thorndike (1942) as well as several 
others mote recently have recognized that even though X and Y are fallible, 
the repression line relevant to ANCOVA is the structural relationship. Since 



6 Y.X * p XX 8 Y-T ’ 

the usual ANCOVA procedures provide and test biased estimates of the treatment 

effects when P v f m . Further, by plugging in different possible values 
• J • j 



of 



J XX» 



Y. T 



and p j it is seen that usual ANCOVA procedures can provide 



non zero treatment effects when the actual effects are zero or zero treat- 
men'; effects when the actual treatment effects are non zero. (Porter, 1967) 
Lord (1960) offers a graphic demonstration of the same problem. 

Lord (1960) was the first to provide a statistical procedure that yields 
and tests unbiased estimates of the correct treatment effects when analyzing 
data from qu^si experiments having a fallible dependent variable and a fallible 
co*vaiiable. His test statistic is asymptotically distributed normal and is 
limited to consideration of only two levels of the independent variable. 

The necessary data are observations on the dependent variable and duplicate 
observations on the covariable, where the duplicate measures follow the 
test-retest paradigm of classical measurement theory. 

I have developed another approach to the prjblem of obtaining and 
testing unbiased estimates of the correct treatment effects when analyzirg 



o 




18 



-18- 



data from quasi experiments having a fallible dependent variable and a 
fallible covariable. At least eompvitat ionally my procedure can be used 
in any design where classical ANCOVA can be used (Porter, 1967 and 1968). 
Essentially the procedure I have suggested and investigated substitutes 
an estimated true score covariable for the observed fallible covariable 
and then employs classical ANCOVA procedures. The estimated true score 
variable in a one way ANCOVA is defined as 

+ P XX < x ij - X .j> > 

For more complex designs the estimated true score covariaole would follow 
the same form except that the observations would be deviated from the 
respective cell means. The important properties o f an estimated true score 
covariable are that it is a linear transformation of the fallible covariable 
and 

1) has the same treatment group and grand means as the fallible 
and unobserved infallible covariable, 

2) has the same correlation with the dependent variable as does 
the fallible covariable, 

and 3) the slope, By j > * s equal to the desired slope of the 
structural relation, By.x • 

From the above three points it; follows that use of classical ANCOVA on 
the fallible dependent variable and the estimated true score covariable will 
provide unbiased estimates of the treatment main effects 

a .j = P Y 1 " “y.. ■ Sy.T^x - »*x..> * 

^nd that the F test statistic will follow the theoretical F distribution 
given the usual assumptions o f ANCOVA plus the assumption that the reliability 
of X is common across all treatment groups. UTien the reliability of X is 
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not known, an estimate mus r bo used which Lntorduros an additional source 
of variation into the model. With the additional source of variation due 
to estimating p , ANCOVA using estimated true scores as the covariable 

AA 

no longer conforms to the classical model and the distribution of the F 
test statistic must be questioned. Also of interest is a comparison of 
the small sample properties of Lord's procedure to the one I have proposed 
when there are only two levels of the treatment independent variable. 

In a recent study (Porter, 1967) I used the Monte Carlo approach M 
investigate the effects of sample size, the reliability of the covariable, 
and the correlation of the dependent variable with the covariable on the 
smaJ 1 sample properties of Lord’s statistic and on the distribution of the 
F statistic calculated from ANCOVA using estimated true scores as the co- 
variable. The ANCOVA procedrre was investigated for both two and four 
levels of the independent variable. Each empirical distribution was based 
on 1000 values vi _he test statistic and the method for estimating relia- 
bility conformed to the test retest paradigm suggested by Lord. Further, 
the covariable was random* 

The results indicated that when the reliability of the covariable was as 
low as .5, the distribution of Lorcas statistic was a vetry poor approxima- 
tion of the normal distribution. As the reliability of the covariable in- 
creased, the number of observations per treatment group necessary for a 
good approximation became less. The size of the correlation of the covariable 
with the dependent variable had an inverse effect on the rapidity of con- 
vergence. Samples of si,*e 20 or greater per treatment group seemed to pro- 
vide for sufficient convergence of Lord’s statistic for intermediate values 
of reliability and correlation. A sample size, larger than 20, was necessary 
in order for the distribution of Lord's statistic to converge upon the normal 
when reliability, correlation, or both were low. 
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The analysis of covariance, using estimated true scores as a covariable, 
also required samples of size 20 or larger per treatment group for tha theo- 
retical F to serve as a useful reference distribution. For covariahlea having 

i 

reliability of .7 or .9, the generated distributions of F based on analysis 
of covariance using estimated true scores as the covaxiable were in close 
agreement with the corresponding theoretical distributions of F. then the 
reliability of the covariable was .5, the agreement of the generated dis- 
tributions of F to their theoretical counterpart^ were not quite as good, 
buL still close. The degree of agreement suffered a greater decrease caused 
by a decrease in the reliability of the covariable for analyses involving 
four treatment groups than for two trea ent groups. For analyses involving 
only two treatment groups the size of the correlation of the dependent vari- 
able with the covariable did not have a systematic effect on the degree of 
agreement of the generated distributions of the F statistic for analysis of 
covariance using estimated true scores as the covariable with the theoretical 
F distribution. However, when using four treatment groups, an increase in 
the correlation of the dependent variable with the covariable cai se a syste- 
matic decrease in the agreement of the generated distribution with the theo- 
retical distribution. 

The analysis of covariance using estimated true scores for the co- 
variable appeared to be os useful a method for testing hypotheses as Lord f s 
statistic. The genera:ed probabilities of a Type 1 Error for two-tailed 
tests were in clore agreement with the theoretical probabilities for both 
test statistics. A slight negative skewness for generated distributions 
of Lord f s statistic caused the probabilities of a Type 1 Error for one- 
tailed tests systematically to exceed trie probrbilit ies of a Type I Error 
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for two-tailed tests. dower was essentially the same for both test statistics. 
The use of estimated true scores was slightly more powerful for the two ex- 
treme levels of reliability, .5 and .9, of the covariable, and Lord’s statistic 
was slightly more powerful for the intermediate level, .7. 

Tne results clearly indicated the utility of Lord’s statistic and my 
procedure of ANCOVA using estimated Lrue scores as the covariable, when 
analyzing data from a quasi experiment where the variables are fallible. 

They also supported the greater generality of th ^ modified ANCOVA procedure. 
Thistletn waite (1969) and Campbell and Erlebacher (1970) provide illustrations 
of the use of ANCCVA using estimated true scores as the covariable. DeGracie 
(1968) has more recently proposed a test statistic which he points out is 
similar to the one that I have proposed and investigated, hut which L.iS an 
asymptotic normal distribution. 

As a final note of caution none of the above mentioned analyses pro- 
vides a completely satisfactory substitution for random assignment of ex- 
perimental units to levels of the independent variable. Although they pro- 
vide and tost estimates of the treatment effects after controlling for 
initial between level differences on tne covariabie, there is no guarantee 
thc:iz tie covariable reflects all important initial between levels dif- 
ferences, i.e., all of the above mentioned procedures suffer from the same 
limitations that apply for ANCOVA on infallible variables. Two excellent 
references on such limitations are provided by Smith (1957) and Lord (1967). 
Elashoff (1969) also points out th^ limitations of ANCOVA when random as- 
signment has not been used in the oosign as well as several other limitations 
of ANCOVA. Informally Lee Crunbach, Donald Campbell and 1 (Campbell and Erle- 
bacher 1970, Errata) have enn^idered the prou .em of choice among covariates 

o 
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where the intent is to use the covariate to control for the problem of con- 
founding variables. Unfortunately our thoughts are at too preliminary a 
stage to be reported here. 

Summ ary 

In this paper problems caused by the existence of errors of measure- 
ment have been identified for factor analysis, regression analysis, ANOVA, 
and ANCOVA. At least one detrimental effect was seen to exist for each 
type of analysis. When a researcher r s interest ^s with infallible variables, 
he rune the risk of biased results from all of the procedures except ANOV r A. 
The estimates of parameters in all four procedures buffer from inflated 
error variance. Some partial solutions were indicated, but clearly more 
work is needed on several of the problems. 

Most statistical procedures have been developed for models where 
variables ere assumed to be free from errors of measurement. Since almost 
all educational research involves use of fallible variables, it is impor- 
tant t fiat the effects of errors of measurement for the various models be 
understood and that the understanding be reflected in current research 
practice . 
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